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Abstract: The mastery or ownership of mathematical language is one view of 
obtaining competence in mathematics. This view defines a profitable 
learning trajectory as one that continually exposes students to the rich 
collection of mathematical expressions that are needed to verbalize and write 
about mathematical situations and the quantitative relationships in those 
situations. The paper describes new steps towards a computerized database of 
addition and subtraction word problems that could provide teachers and 
students with access to critical natural language terms of and expressions for 
mathematical relationships. On the basis of new results from a computer 
simulation that is sensitive to slight changes in problem wording, the 
suggestion is that a database of word problems that reflects a relative order of 
difficulty is not only feasible but essential if researchers in this area are to 
increase the practical dissemination of research results. A collection of rtath 
stories sensitive to children's text comprehension skills and mathematical 
development is the foundation of an ongoing effort to implement a 
knowledge-building environment that can facilitate childrens' discussions of 
why one math story is more difficult than another as well as encourage 
communities of students to build their own database of stories. 
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Introduction 

Relating natural language to mathematical language is an important 
component of elementary mathematics education. The mediating role of 
natural language in expressing everyday problem situations is increasingly 
being recognized as a potential focus for both research on children's 
mathematical problem solving and new classroom approaches which 
encourage children to construct their own solutions (Steffe, Cobb & von 
Glaserfeld, 1988). The NCTM Curriculum Standards (1989) emphasize the 
pedagogical importance of representing and reading mathematics, and both 
the Curriculum Standards and the Professional Standards (1991) focus on 
encouraging children to write mathematics and particularly to talk about 
mathematical aspects of situations. Observing how children make their 
thinking about such situations explicit in terms of natural language also has 
value for purposes of assessment (Carpenter & Fennema, 1988; Loef, Carey, 
Carpenter, & Fennema, 1988). Some kind of verbalization of mathematical 
problems enters into all of these activities. 

Traditionally, the verbal aspect of mathematics at the elementary level has 
been incorporated into curricula as "word problems/ 7 which have often 
served as a source of children's continuing frustration with mathematics. 
The recent emphasis on how children could or do make connections between 
natural language and mathematical concepts in discourse, whether they 
proceed from a given verbal representation or use a verbal representation to 
make precise a problem situation, provides new motivations for a 
consideration of why certain linguistic forms of mathematical problem 
situations are difficult. Carpenter, Fennema, Peterson, and Carey (1988) note 
that most teachers' knowledge of children's thinking is not organized into a 
coherent network which distinguishes different types of problem difficulty. 
In addressing this fact, the suggestion inherent in this paper is that teachers 
would benefit from a computerized database of problems which reflects the 
relative difficulty or facilitating effect of phrasing of those problems. In a 
related manner, children's linguistic expression of problem situations might 
reveal aspects of their comprehension to a teacher who is sensitive to the 
competence represented by certain means of expression. In attending to 
different types of language and linguistic understanding, the proposed 
approach to type? of problem difficulty goes beyond the traditional problem 
c ossification by semantic structure, considered by some researchers (e.g., 
Cobb, Yackel & Wood, 1988) to be an incomplete form of pedagogical 
know edge. 

The idea of focusing on various cognitive dimensions of word problem 
solution rather than simply on problem classification has received support 
from interdisciplinary research concerned with, e.g., the recognition of 
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semantic relationships (Booth, 1989; Herscovics, 1989; Kaput, 1987; and many 
others) and the use of problem-solving strategies (Thompson, 1988; Greeno, 
1987) based on understanding as opposed, for example, to superficial "clue- 
word" approaches. However, linguistic factors (Cummins, 1991; Lewis & 
Mayer, 1987; Stern, 1993 and others) and presentational factors (Davis-Dorsey, 
Ross & Morrison, 1991; De Corte, Verschaffel, & DeWin, 1985; Staub & 
Reusser, 1992) have only recently been recognized as a level of representation 
at which misunderstandings can occur. These results give insights into 
isolated sources of difficulty, as do informal classroom observations of 
children's mathematical discourse. Such approaches highlight the 
importance of understanding specific "cognitive bottlenecks" as a critical step 
towards designing effective classroom intervention strategies (e.g., Carpenter 
& Fennema, 1988; Resnick, 1992). 

However, there is as yet no detailed model of children's complete solution 
process (that is, beginning with a left-to-right reading of individual sentences) 
for all problem types (e.g., Change, Combine, Compare), although there is 
ongoing work which begins to address this (Fuson, 1994b; Reusser, 1990; 
Weber-Russell, 1994). Most cognitive simulations of word problem solution 
(Briars & Larkin, 1984; Cummins, Kintsch, Reusser, & Weimer, 1988; Kintsch 
and Greeno, 1985; Okamoto, 1994; Riley & Greeno, 1988) start directly with 
artificial propositions. The exception is Reusser' s Situational Problem Solver 
(SPS) (1990). SPS possesses an elaborate text-processing component to 
distinguish the representation of the situation from the representation of the 
text, as advocated by Kintsch (1986). SPS is thus the first attempt to simulate 
the progressive and incremental process of transformation from text to 
situation to equation. In particular, SPS stresses that from a problem solving 
(and instructional!) point of view, arriving at a representation of the situation 
in a problem should not be a superfluous, but a necessary process. SPS does 
not however address static and/or relational language problems, e.g., 
Combine and Compare types. Thus, the role of situational representations in 
statically-worded problems (i.e., problems which do not contain significant 
action language) is not entirely clear (cf. Stern & Lehrndorfer, 1992). 

The simulation and results presented in this paper address the dual need for a 
model which starts with the process of reading as well as one which 
recognizes the critical role of situational representations in statically worded 
texts. Centered around results from this "bottom-up" computer simulation 
which is sensitive to changes both in problem wording and presentational 
structure (LeBlanc & Russell, 1993), the case is presented that the time is right 
to establish a computerized database of arithmetic and subtraction word 
problems that includes a multiple number of rewordings for each 
"traditional" problem wording. Predictor equations from regression analyses 
are used to confirm previous research on the effects of problem wording and 
to extend the traditional measures of why one problem is easier or harder 
than another. The premise is that a synthesis of research results concerning 
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(i) children's developing mathematical competence; (ii) developing 
lin guistical competence in conjunction with (iii) knowledge of the affects of 
problem wording in the form of a 'database cart provide real-time 
instructional suggestions as to the "next best problem to present/ 7 

Specific research questions of interest include: 

• What kinds of natural language used to convey a problem situation have 
different effects on the ability to conceptualize the problem in terms of 
correct mathematical relationships? and 

• Assuming that problem sequencing is important in order to encourage 

progress without making jumps for which the student is not ready, does 
sequencing word problems (for example, in the form of a database) suggest 
a more profitably learning trajectory for students? 

The paper proceeds as follows. After a brief introduction to the computer 
simulation, empirical results are presented which show how the simulation 
defines word problem solving difficulty as a function of memory load and 
text integration inferences. Three traditional word problem types are then 
reviewed, including examples which show multiple ways to reword those 
types of problems. Where possible, empirical studies, including results from 
the computer simulation, confirm and /or suggest a relative order of difficulty 
between multiple rewordings of problems. 

The Computer Model 

The current computer simulation comprehends arithmetic word problems 
written in English in a word-by-word, sentence-by-sentence, "bottom-up" 
fashion. The simulation is composed of Lwo components: EDUCE and 
SELAH. EDUCE is an expectation-driven parser adapted from the work of 
Schank and Riesbeck (1981) and developed according to principles found to be 
distinct for word problem solving (Burns, 1993; LeBlanc and Russell, 1989). 
EDUCE handles a number of language features inherent in word problems, 
including: (i) quantities (e.g., 3, three); (ii) noun compounds (e.g., soda cans)) 
(iii) pronominal reference (e.g., she)) (iv) time sequence (e.g., then, in the 
beginning) (v) set partitions (e.g., [by ownership: "Dick and Jane have"], [by 
type of object: "8 cats and dogs"]); (vi) ellipsis (e.g., "Ed has 3 cans. He found 2 
more." That is, 2 more cans) and (vii) reference to previous sets (e.g., 4 of 
them). SELAH is a text integration component which accepts EDUCE's 
canonical representations of individual sentences and constructs a situational 
interpretation of the problem based on explicit conceptual actions in the text 
or direct or implicit references between sets (LeBlanc, 1991). While building a 
"situation model," SELAH instantiates an arithmetic action and later a 
counting strategy to solve the problem. 

5 
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The simulation is currently implemented as a "bottom-up" problem solver. 
SELAH completely determines the role of each sentence after it is read by 
EDUCE and then the entire reading and text integration process continues to 
the next sentence. Bottom-up processing is opposed to a more global, expert- 
like, "top-down" strategy that potentially involves an instantiated schema in 
memory for each particular type of problem and/or an associated plan to look 
for quantities which can fill slots in that schema. Although SELAH has been 
designed so that it could function in a more expert-like top-down fashion, the 
focus of the current implementation is a model of the young reader or, at 
least, a novice word problem solver. At this stage, the simulation is not a 
developmental model in the sense of Briars and Larkin (1984), Riley et al. 
(1988) and Okamoto (1994). That is, in the current implementation, SELAH 
does not alternate between varying levels of expertise. More specifically, the 
focus is to model the necessary text comprehension and logico-mathematical 
requirements of problem solution. Because SELAH solves all of the problems 
in the Riley et al. (1988) benchmark set, it must exhibit expert-like capabilities 
(e.g., performing a relatively large number of inferences) on some problems 
but only rudimentary capabilities on the easiest problems. For example, some 
problems involve higher memory loads than other problems. While it is 
possible to set a limit on the number of concepts which can be stored in 
working memory (c/. Fletcher, 1986), tha simulation currently solves each 
problem with an unlimited memory capacity. One theoretical interest is the 
total demand on memory that each problem requires when read in a bottom- 
up fashion. In short, EDUCE and SELAH simulate a problem solver who 
reads each sentence only once and attempts to maintain the sets and their 
relationships in memory until deciding on an appropriate arithmetic action 
which can lead to a solution. 1 

A unique feature of the simulation is that it keeps track of (i) the number of 
text integration inferences that are required and (ii) the load on working 
memory while performing text integration across sentences. The 
fundamental hypothesis of the simulation is that children's ability to follow- 
up on explicit set references (or infer such references) is a crucial step towards 
recognizing the conditions that make an arithmetic operation appropriate for 
a given situation. The empirical results present new measures of text 
comprehension which determine why a particular word problem oay be 
difficult, especially for young readers. 



1 The current implementation of SELAH does not simulate solutions which use manipulatives 
such as blocks on the table (c/. the CHIPS model of Briars & Larkin, 1984). In a sense, children 
who use manipulatives are closer to a strict definition of bottom-up problem solving; that is, for 
each sentence that is read the blocks are moved. The focus of SELAH is to model the total text 
comprehension requirements that novices experience as they solve these problems, including the 
cognitive demands of remembering previous sets rather than off-loading those requirements by 
depending on physical aids. 
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Example of the Simulation Reading and Solving a Problem (see Appendix) 

A detailed example of how the parser, EDUCE, is sensitive to changes in 
problem wording and how SELAH builds a situational representation and 
solves a problem is given in the Appendix. 

Summary of Experimental Results 

In a series of exploratory regression analyses, children's probability of solution 
on a benchmark set of problems from Riley et al (1983, 1988) were correlated 
with the monitored performance of the simulation on those same problems. 
Global measures of text integration and local sentence-level variables which 
account for a significance proportion of the variance in children's probability 
of solution were isolated. (A thorough presentation of the experimental 
method and results are beyond the scope of this paper and would detract from 
the focus on problem rewording. The interested reader is directed to LeBlanc 
and Russell (1993) for an abbreviated view and/or LeBlanc (1993) for a 
complete discussion). 

One significant result emerging from the experiments deserves mention 
here. A combined measure of the number of concepts to remember and the 
number of inferences to make while remembering those concepts is a 
consistent predictor of children's problem solving success (for all grades K 
through 3, the variance accounted for (R 2 ) is at least 50%, p<.001). If one 
considers a child's total processing space as fixed (as suggested by Case, 1982), a 
word problem that requires a relatively large number of concepts to be held in 
memory also limits the resources that can be devoted to executing basic 
operations, including the making of inferences. In the context of arithmetic 
word problems, the inferences necessary for text integration are critical. An 
ability to arrive at a correct arithmetic operator is directly dependent on 
establishing the relationships between sets, which in turn is dependent on an 
integrated representation of the text. Unlike narrative texts, word problems 
tend to be very brief and consecutive sentences contain limited degrees of 
overlap (c/. Haviland & Clark, 1974). Children who are unable to make the 
inferences which lead to "mathematical connections" are confronted with 
independent sets in memory and must resort to ad hoc strategies (e.g., a 
keyword or first-number-given strategy) as confirmed in the literature 
(Cummins et al, 1988; De Corte & Verschaffel, 1985; and others). 

The empirical results suggest that children's difficulties with arithmetic word 
problems are due in part to an inability to make text integration inferences, 
especially when a relatively high number of concepts occupy memory. The 
implications for instruction are two-fold. First, the processes of text 
comprehension and mathematics are tightly coupled in arithmetic word 
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problem solutions; however, there are fine-grained methods of altering a 
problem's probability of solution in each area. Because making 
"mathematical connections" is so critical, these results suggests that 
rewording problems in ways which imply or even explicitly state the 
relationships between sets is a critical step towards helping those students 
who can not yet make the necessary inferences, as recently shown by Davis- 
Dorsey et ah (1991) and others. Children are often expected to make complex 
inferences required by sparsely worded problems while they are just 
beginning to read. The second educational implication, related to the first, is 
that a more fine-grained classification of word problems is emerging. 

Toward a Database of Word Problems 

In conjunction with developmental theories of children's addition and 
subtraction competencies (Fuson, 1994b; Okamoto, 1992), a number of task 
characteristics individually and in combination have been shown to affect the 
solution probability of arithmetic word problems: 



• semantic structure; e.g., change problems are generally easier than compare 

problems (Riley, Greeno & Heller, 1983; Carpenter, 1985; De Corte, 
Verschaffel & Pauwels, 1990; and others) 

• misunderstandings of quantification or reference; e.g., "some" is an 

adjective like "red", or "altogether" means each (Cummins, Kintsch, 
Reusser, & Weimer, 1988) 

• complex relational phrases; e.g., "more than" and "less than" (Stern, 1993; 

Hudson, 1983 and others) 

• "inconsistent" language; e.g., problem says "less" but requires addition 

(Lewis & Mayer, 1987 and others) 

• qualifying dependencies; supersets are partitioned based on ownership (Jim, 

Sally, Jim and Sally), adjectives (e.g., big, small, big and small), arguments 
(e.g., boys, girls, children), and location (on the table, on the shelf, in the 
room) (Nesher, 1982) 

• structural format; sequence of the given numbers: problems start with the 

smaller vs. the larger number (Verschaffel & De Corte, 1990); extraneous 
information (Searle, Lorton & Suppes, 1974); location of the unknown: 
e.g., unknown is the initial set (Hiebert, 1982) 

• general context and affective factors; e.g., use of favorite objects and the 

names of friends (Davis-Dorsey, Ross & Morrison, 1992; c/. McLeod, 1988) 



AERA-94 



• presentational structure; time sequence: sentences are not in chronological 

order; narrative focus: protagonist is not always in actor role; related co- 
actors: one person is referred to in terms of her family relationship to the 
other person (Staub & Reusser, 1992) 

• text integration; absence of potentially helpful explicit set reference 

language): e.g., "of them", "more", "the rest" (De Corte, Verschaffel, & 
DeWin, 1985; and others); integrated propositions in memory (O'Brien, 
1987; Trabasso & Sperry, 1985; van den Broek, 1988); importance of short- 
term memory as a bottleneck in the comprehension process (Cooney & 
Swanson, 1990; Fletcher, 1986; LeBlanc, 1993). 



The next sections introduce example problem types where the presence or 
absence of specific task characteristics influence solution probability. Because 
the simulation is sensitive to slight changes in problem wording, the 
monitored performance of the simulation for a particular rewording can be 
entered into the predictor equations that were derived from the regression 
analyses. Other empirical studies are cited which show a relative order of 
difficulty between two problems and the predictor equations are used to 
confirm the empirical results and/or predict new effects of a task characteristic 
on the probability of solution. 

Making Static Non-Relational Problems Easier 

The following problem is typical of a class of problems which do not contain 
significant actions (thus the term, static) and do not involve relational 
expressions such as "more than" (thus the term, non-relational). (Riley et al 
(1988) refer to this particular problem as Combine 5 - subset unknown and 
Fuson (1994a) refers to it as Put Together - unknown part .) 




Problem 1 - Traditional (Combine 5) Wording 



In this problem, the first sentence describes a superset with a known amount. 
A subset with a known quantity is then described, followed by a question 
requesting the amount of the other subset. 

This problem has received considerable attention in the literature, mostly due 
to its level of difficulty. Considering all eighteen problems in Riley's (1988) 
benchmark set, this problem, on average, is the third most difficult problem. 
Children's success rates across the first four grades in the Riley et al (1983, 
1988) studies reflect the difficulty: 22%-K, 33%-lst, 55%-2nd and 75%-3rd. 
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These relatively low probabilities in even the higher grades are confirmed 
elsewhere (Davis-Dorsey et al, 1991; De Corte, Verschaffel et al, 1985). 

There are two potential r^wordings which might help those students who are 
not able to solve this type of problem: (i) the use of language to facilitate text 
integration between sentences and thereby highlighting the relationship 
between quantities (e.g., of them, the rest), see Problem 2 and (ii) the same 
changes in (i) along with the removal of the conjunction {and) and the 
removal of the word altogether , see Problem 3 . 

The first potential alternative (Problem 2) to the "traditional" wording 
(Problem 1) is to include words which facilitate the process of text integration 
by highlighting the relationship between quantities. 



^ of 2^^^' 

When EDUCE parses the second sentence of Problem 2, the phrase "of them" 
in the second sentence generates an explicit reference to a previously 
mentioned set, i.e., an explicit reference to David and Kathy's set of 8 (see the 
Appendix for a complete discussion of how the simulation handles this 
problem). This explicit reference identifies that David's 5 are part of the 
previously mentioned set of eight . In other words, EDUCE is sensitive to the 
"mathematics embedded in the natural language." On the other hand, in the 
second sentence of Problem l's traditional wording ("David has 5 soda cans"), 
no reference to the previous set of eight is made in the text. Thus, EDUCE' s 
representation of the second sentence of Problem 1 does not contain an 
explicit link back to the first sentence. When performing text integration, 
SELAH must make an inference to establish that David's five cans are part-of 
David and Kathy's original set of eight can s, as opposed to five totally 
unrelated cans. 

In addition, the phrase, "The rest of them" in the third sentence of Problem 2 
marks Kathy's set as part of thj eight soda cans that are left as the result of a 
previous separation . In the traditional (Problem 1) wording, no reference is 
made to Kathy's set so SELAH must infer that Kathy's cans are part of the set 
of eight as well as those remaining after separating out the cans belonging to 
David . 

De Corte, Verschaffel et al (1985) and others have provided empirical support 
that the reworded version significantly increases children's solution 
probability. In line with these results, the simulation predicts that including 
the phrases "of them" and "the rest" will increase solution probability. The 
average memory loads for these two problems are quite similar although the 
simulation performs three fewer inferences on the reworded version. 
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Table 1 compares first and second grade childrens' success on both the 
traditional and reworded versions as reported by De Corte, Verschaffel et al. 
(1985) and predicted by the simulation. As might be expected, the simulation's 
predictor equation for first grade is more sensitive to a change in the number 
of inferences than a later grade, although for both grades (1st and 2nd), the 
simulation predicts a greater increase in solution probability (from the 
traditional to the reworded version) than found by De Cor<-e et al 



Table 1 

Fredicted and observed 1st and 2nd grade probabilities 
for versions Problem 1 & 2 



Problem 
Wording (grade) 


Simulation's 
Prediction 


De Corte 
Data 


Traditional (1st) 
Problem 1 


0.28 


0.43 


"altogether" & 
"of them" (1st) 
Problem 2 


0.59 


0.57 


Traditional (2nd) 
Problem 1 


0.55 


0.71 


"altogether"" & 
"of them" (2nd) 
Problem 2 


0.81 


0.83 



Another potential rewording (see Problem 3) involves the use of the phrases 
"of them" and "the rest" like Problem 2, yet the conjunction ("and") and the 
word "altogether" in the first sentence have been removed. 

im -^i I \ ? *V\me are Bjsod^ to David, \ • 

; ; : The rest are Kathy's* How many soda cans does Kathy have? 

Problem 3 - No "and/altogether" Rewording of Combine 5 

Cummins (1991) used this version in her studies of the factors that influence 
childrens* interpretations of arithmetic word problems. According to 
Cummins, the critical difference between this rewording and the other 
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rewording (Problem 2) is the absence of the conjunction in the first sentence 
(the removal of "altogether" is considered secondary). A conjunction of 
owners (David and Kathy) is often misinterpreted by young children as 
meaning each. Thus, in a sentence such as: 

David and Kathy have 8 soda cans altogether. 

Many children misinterpret this to mean: " David has S nnd Kathy has 8 ." 
Such a misunderstanding would lead children to answer "8" when asked 
how many Kathy has. Cummins validates her hypothesis with results from a 
computer simulation, childrens' recall protocols (Cummins et ah, 1988) and 
the classification of solution error types (Cummins, 1991). In the latter study, 
children were found to commit given-number errors (i.e., the answer to the 
traditional wording is "8," a number given in the problem) on 46% of the 
incorrect response? (see De Corte, Verschaffel et ah, (1985) for similar results). 

Table 2 compares first grade children's success on both the traditional and 
reworded versions as reported by Cummins (1991) and De Corte, Verschaffel 
et al (1985) and predicted by the simulation. 



Table 2 

Predicted and observed 1st grade probabilities 
for reworded versions of Problem 1 (Combine 5) 



Problem 
Wording 


Simulation's 
Prediction 


De Corte 
Data 


Cummins 
Data 


Traditional 
Problem 1 


0.28 


0.43 


0.30 


"altogether" & 
"of them" 
Problem 2 


0.59 


0.57 




no "and" 
no "altogether" 
Problem 3 


0.75 




0.85 



The simulation offers an information processing perspective as to why many 
children use the wrong operation in their solution (i.e., they add 8 and 3 
rather than subtracting 3 from 8). According to the simulation, the critical 
advantage of Problem 3 is the removal of the word "altogether." In the 
simulation, the parser performs three tasks upon reading the word altogether: 
(i) partition the current set, although the qualifier which causes the partition 
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(e.g., different owners, different colors) is currently unknown; (ii) attempt to 
determine the qualifying dependency that partitions the set; and (iii) 
instantiate the arithmetic action Join. 2 In the traditional wording, after the 
first sentence is read, the simulation has the following representation in 
memory: ( David & Kathy's 8 soda cans, TOIN) . While the "sets to join" are 
still unknown, the critical point is that the Join action has been activated. 
Upon reading the second sentence, the simulation infers that David's 5 are 
part of the previous set of 8 . In order to construct David's 5 from the set of 8, 
the arithmetic action of Separating-From is needed. Thus, as the simulation 
reads the traditional wording of Problem 1, the Join action must be 
suppressed in favor of the Separating-From action. The simulation suggests 
that young children may not be able to suppress previously activated 
information, thus they remain with the initial Join activation. Given the 
Cummins (1991) reworded version without the presence of altogether in the 
initial sentence, the simulation is not required to suppress an initial 
activation of Join. 

Making Si gnificant Action Languag e Problems Harder 

In addition to being sensitive to wording changes which make problems 
easier, the simulation also confirms empirical results for rewordings which 
cause problems to be more difficult, for example, altering the time sequence of 
problems containing significant action language. (Riley et al. (1988) refer to 
significant action problems as Change join and Change Separate problems 
and Fuson (1994a) refers to them as Change Add To and Change Take From .) 

As shown by Staub and Reusser (1992), altering the time sequence of a Change 
problem such that the sentences are not in z'aict chronological order makes 
the problem more difficult. Problem 4 uses the traditional wording of a 
Change 1 (Change Add Tc? problem: 

• "^(Alexhad 5 marbles. Then Bethany gave Alex 3 marbles. " 
Ho w many marbles does .Alex have now? "■■ ; 
Problem 4 - Traditional Change 1 Wording 

As shown in Problem 5, a change to the presentational structure causes the 
first sentence to refer to the transfer of possession action and the second 
sentence to refer to the initial set that existed prior to the action: 

ippBlil^ffix^aibles. Yesterday'. Alex- had 5 marbles. 

I How many marbles does Alex have now? 

Problem 5 - Altered Time Sequence of Change 1 



2 In the simulation, instantiating an arithmetic action of Join is not analogous to picking the 
arithmetic operator of addition. The Join action simply means that the current set is 
partitioned and can be formed by joining two other (perhaps as yet unmentioned) sets. 
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A critical difference between the traditional wording and the time-altered 
version is explained through the simulation when it solves these two 
problems. In the first sentence of the traditional wording (Problem 4), Alex is 
understood to possess five marbles and the simulation creates a set of five for 
Alex. The second sentence is understood to be a transfer to Alex, thus the 
three that are transferred are JOINed to his previous five as described by the 
"transfer-in" situation in the first two sentences. The JOIN action is explicit in 
the text and subsequent situational representation and need not be inferred. 

In the time-altered version (Problem 5), the transfer of three to Alex occurs in 
the first sentence. Because Alex does not previously have any in his 
possession, the program simulates the transfer as resulting in Alex's set of 3 
today. The second sentence indicates that Alex had 5 yesterday and the 
simulation makes another set for Alex with this amount. Since the program 
is simulating bottom-up, sentence-by-sentence reading, it does not "rerun" 
the first two sentences over in a chronological fashion. After interpreting the 
third sentence to mean a request for the number of marbles that Alex has 
now, the simulation infers that a difference in time differentiates the three 
sets so it can JOIN Alex's marbles-fodny and Alex's marbles-yesterday . In 
short, when the simulation comprehends the time-altered problem, the 
transfer of 3 from Bethany to Alex in the first sentence does not instantiate a 
JOIN action. That action must be inferred based on the qualifying dependency 
of time (today, yesterday, now) at the end of the problem. 

Staub and Reusser (1992) have clearly shown that the time-altered version of 
Change 1 is more difficult to solve than the traditional version. Because the 
simulation is sensitive to the number of inferences that are required in each 
of these versions, the predictor equations verify their results. Table 3 presents 
children's solution probability with the traditional wording (Riley & Greeno, 
1988), children's solution probability with the time-altered wording (Staub & 
Reusser, 1992) and the simulation's predicted solution probabilities. 



Table 3 

Predicted and observed 1st grade 
probability of solutions for Change 1 



Problem 
Wording 


Simulation's 
Prediction 


Staub 
Data 


Riley 
Data 


Traditional 
Problem 4 


0.90 




1.00 


Time-Altered 
Problem 5 


0.70 


0.63 





13 ,4 



AERA-94 



As expected, the simulation closely predicts the solution probability for the 
traditional wording. 2 In addition, the simulation does predict a decrease in 
solution probability (70% success rate) but not as much of a decrease as Staub 
and Reusser found (63%). One possible reason may be that- some children in 
the Staub study are confused by the difference between today and now, 
whereas the simulation is not. If this were the case, a number of children 
would be expected to give the answer of three (the number Alex had today). 
Staub reports, however, that only a small percentage of children who get it 
wrong give this answer (personal communication). Another possible reason 
from the cognitive simulation perspective is that one or both of the 
inferences required in the time-alteration version may be very difficult 
inferences for children to make. For example, the inference that marks the 
set [Alex has ? now] as the superset would appear to be a complex one; i.e., the 
word "now" is the primary clue that the sets should be distinguished by time. 
Distinguishing superset and subsets by time is in fact a more abstract type of 
the more difficult Combine (Put Together) problems, where sets are typically 
distinguished by ownership (John has, Mary has, John and Mary have) or 
other features such as color (red marbles, blue marbles, red and blue marbles). 
Because the simulation counts all inferences with the same weight (i.e., +1), 
the simulation may be underestimating the difficulty of certain types of 
inferences, such as those due to semantic character (e.g., modality, degree of 
abstraction of objects and relationships). This is one indication that the types 
of inferences may be at least as important as, if not more important than the 
number of inferences. 



Making Static Relational Problems Easier 

Static (no significant actions) relational (comparative language) problems are 
the most difficult addition and subtraction problems, as convincingly 
confirmed in empirical tests of childrens' solutions and recall protocols 
(Cummins, 1991; Fuson, 1992; Lewis & Mayer, 1987; Stern, 1993), eye- 
movement experiments (Hegarty, Mayer & Green, 1992; Verschaffel, De Corte 
& Pauwels, 1992) and studies of childrens' use of relational terms in the home 
(Walkerdine, 1990). Problem 6 is typical of this class of problems, commonly 
referred to as Compare problems. 



9 
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2 The simulation is expected to closely predict the solution probabilities on the traditional 
wordings because the predictor equations were derived from childrens' solution probabilities 
when they solved problems with the traditional wordings. Note, however, that the equations 
are the result of an analysis using all three types of semantic structure, not just Change 
problems. 
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• 'f:;^ soda cms. Oiiim^^oti 

How many soda cans does Jacob have more tlrm 



Problem 6 - Traditional Wording of Compare 1 

In Problem 6 (a difference set unknown problem), the cardinalities of two 
disjoint sets are compared. The question requests the difference between the 
sets with a focus on the larger set, i.e., "how many more?" One possibility for 
making the traditional wording of Problem 6 easier is to replace the static 
relational language (e.g., "more than") with action language which describes a 
hypothetical act of making two sets equal (Carpenter, Hiebert & Moser, 1981 
and others). With this "equalize" language, the static relational language (e.g., 
"how many more than") in the traditional wording of Problem 6 can be 
replaced with action language which describes a hypothetical act of making 
the two sets equal (e.g., "how many does one need to find to have as many as 
the other"), as shown in Problem 7. 



• Jacob Jhas 2 soda cans/ Chxissy has 8 soda cans, 
, : 2 - How many soda cans does Jacob need to find 

Problem 7 - Equalize Rewording for Compare 1 

For all grade levels, the simulation predicts that the Equalize version will 
improve solution probability, especially for the youngest grades, due largely to 
the fact that the Equalize version provides an explicit action to Join ("need to 
find") and an explicit reference to the result of that Join ("as many as 
Chrissy") On the other hand, the simulation must make these two inferences 
when solving the situationally void traditional wording. Table 4 compares 
the solution probabilities of the traditional Problem 6 as found by Riley et ul. 
(1988), the Equalize probabilities as found by Carpenter et al (1981), as well as 
the simulation's predicted solution probabilities of the traditional and 
Equalize rewording (Problem 7). 

Table 4 

Comparison of predicted and observed Compare 1 
probabilities and the Equalize version for 1st Grade 



Problem 
Wording 


Simulation's 
Prediction 


Carpenter 
Data 


RUey 
Data 


Traditional 
Problem 6 


0.48 




0.28 


Equalize 
Problem 7 


0.90 


0.91 
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Future Work 



At the beginning of this paper, the question was asked: 

[Does sequencing word problems (for example, m One form of a database) 
suggest a more profitably leaimmg trajectory lot students? 

Answer #1: No, if the sequencing leads students toward easier problems and 
allows them to avoid the difficult problems. In fact, if sequencing in the form 
of a database is to have any merit, it must expose students to problems that 
are more difficult than those found in standard textbooks in the United 
States, as argued by Fuson (1994a). Sequencing must specifically challenge 
students to deal with complexity in situational descriptions in text of the kind 
emphasized in algebra instruction (Thompson, 1993). 

Answer #2: Yes, if the sequencing leads students to participate in community 
discussions about why some problems are more difficult than others. 

Answer #2 is the motivation of our current work. 3 While sequencing 
problems at levels of detail beyond semantic structure has advantages for the 
development of questions for text books and achievement tests, the 
immediate concern is to combine the wealth of linguistical, mathematical, 
and developmental knowledge with tools which both consolidate research 
results for classroom teachers and encourage student discussions abr^ut and 
ownership of mathematical language. We believe this is the type of software 
support that can "foster efficient inquiry" in the classroom (Kaput, 1992). For 
example, students (and teachers!) could benefit from an environment which 
encourages them to: 

(1) rank order a set of problems according to predicted levels of difficulty 

(2) empirically verify within their own classroom the rank order of difficulty 

(3) compare the predictions in (1) with the results of (2) and discuss 

(4) compare the results of (2) with an "expert" database and discuss 

(5) write their own math stories which include the "difficult" language 

(6) create their own publicly available "math text" to share on the network 



3 In a second grade class at Merrymount Elementary School in Quincy, Massachusetts, we are 
engaging students in mathematical discourse by allowing them to rank-order word problems 
according to both their own predicted and in-class-verified orders of difficulty. 
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From a software engineering perspective, the construction of a working 
database of problems is not dependent on the continued development of the 
simulation (no small task!), although an "expert" reader will be a necessary 
component for an environment which allows teachers and students, to add 
their own problems to a database, as hinted at in (5) and (6) above. 

If the mastery or ownership of mathematical language is a valid view of 
target competence (and obviously from the above, I believe it then a 
profitable learning trajectory is getting students to a point where they can 
verbalize and write about mathematical situations and the relationships in 
those situations. A database of problems would appear to be the next logical 
(perhaps essential?) step. What do you think? 4 



If you don't catch me in New Orleans, I'm always on-line: Mark_LeBlanc@wheatonMA.edu 
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Appendix 

Example Simulation of EDUCE and SELAH 
Solving a Static-NonRelational (Combine 5) Problem 

The aim of this computer simulation is not to translate directly from the 
problem statement into mathematical notation (commonly referred to as a 
problem model), but rather to make precise the tacit steps and processes 
which mediate a translation. Starting from the beginning, that is, the input to 
the simulation is identical to that which children see, the simulation "reads" 
and represents each word and sentence in the problem. Arithmetic actions 
are selected on the basis of the explicit or implied situational relationship 
between integrated sets, and final solution processes simulate the tasks of 
arriving at and carrying out an appropriate counting strategy. Reading 
comprehension overlaps with mathematical comprehension. 

The implementation of these steps in a computer model reflects a particular 
text comprehension bias that deserves mention. Reading word problems 
requires a set of domain specific text comprehension strategies that are not 
utilized in the process of reading other types of texts, as pointed out by 
Kintsch and Greeno (1985). For example, in natural language, numbers 
function as predicates of objects whereas in word problems, quantified noun 
groups must be abstracted to the point where the numbers are the objects of 
interest (Nesher and Katriel, 1986). Researchers refer to this ability to 
translate between and/ or abstract from natural language and the formal 
language of mathematics in a number of ways: playing the "word game" 
(DeCorte and Verschaffel, 1985), understanding "textual presuppositions" 
(Kintsch and Greeno, 1985), and possessing the "mathematics register" 
(Spanos, Rhodes, Dale and Crandall, 1988). 

Implementation Details 

The computer model software is written in Common Lisp. The' EDUCE 
component is implemented as an expectation-based system (somewhat 
analogous to a traditional rule-based system) and totals approximately 23,000 
lines of Lisp. The SELAH component is implemented in a procedural fashion 
and comprises about 9,300 lines of Lisp. The user has the option to run the 
model in interpretive fashion (i.e., load uncompiled code for development 
purposes) or first compile the Lisp. The size of the model when compiled is: 
EDUCE files (except external lexicon): 262K; SELAH files: 152K. Because the 
implementation has strictly followed the Common Lisp standard, the model 
executes on a number of platforms which support Common Lisp, including a 
RISC mainframe (Lucid Common Lisp /DECsy stem, v4.0), SUN workstations 
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(Sun Common Lisp, v4.0.1) and a number of Macintosh computers 
(Macintosh Common Lisp; requires at least 4.5 MB disk space and 2 MB 
RAM). 

An Example 

Some of EDUCE's sentence-level reading capabilities and SELAH's text 
integration processes are revealed below in the annotated script of the 
simulation's output while solving the following word problem: 




As each sentence is parsed, EDUCE passes its conceptualization of that 
sentence to SELAH. (I will defer a detailed discussion of how EDUCE parses 
sentences until the more interesting second sentence). Because EDUCE has 
not detected any pronominal or set reference in the first sentence, SELAH 
knows that there is no explicit request for text integration. No (implicit) 
integration is possible, since no other conceptualized sets currently reside in 
long-term memory (LTM) 4 , so SELAH recognizes the conceptual possession 
of a quantified object as a set and stores the new set in LTM: 

LTM-(l) 

STATE: possession 

POSSESSOR: (Jacob Kathy) 
SET: 

OBJECT: (physical (-animate) (function: contain(object: soda))) 
QUANTITY: 8 

At this point, control returns to EDUCE and the second sentence is parsed. 
Each word in the second sentence is read one at a time starting from left to 
right. Upon reading the first word, "Jacob" EDUCE recognizes this word as a 
name and performs a referential search to see if Jacob has been previously 
mentioned. Since Jacob has been previously mentioned (in the first 
sentence), the already existing concept for Jacob is accessed and its conceptual 
meaning is loaded into short-term (working) memory (STM): 

CONCEPTS: (1) PERSON - (physical (+human) (reference: Jacob)) 

REQUESTS: nil 



4 The convention of referring to long-term memory (LTM) as a "location" serves as a convenient 
method of speaking of "level of activation." LTM represents those concepts which have 
presumably "decayed" to a state whereby reactivation is necessary in order to reference them. 
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There are no expectations or requests generated by the word Jacob, i.e., the 
reading of this word does not expect (or request) any particular concepts to 
follow. The word "has" is then read and the conceptual entry for the word has 
is loaded into STM (2) along with its associated requests: 

CONCEPTS: (1) PERSON - (physical (+human) (reference: Jacob)) 
(2) STATE: possession 
POSSESSOR: ? 
OBJLCT: ? 

REQUESTS: (i) Who possesses? [search for human earlier in sentence] 
(ii) What is possessed? [search for an object later on] 

Before reading the next word, each of the requests associated with the word 
has is tested to see if it might be satisfied. The first request (i) is of course 
successful since a search finds the conceptual representation of Jacob in STM 
(i.e., Jacob's conceptualization is +human) and thus the (1) Jacob-concept is 
merged into the POSSESSOR-slot of the (2) STATE-possession-concept. The 
second request (ii) is unsatisfied since no conceptual-object currently resides 
in STM. Having read and completed the processing for the first two words, 
"Jacob has," EDUCE has one concept and one outstanding request in STM: 

CONCEPTS: (2) STATE: possession 

POSSESSOR: ((+human) (DEFINITEref: Jacob)) 
OBJECT: ? 

REQUESTS: (i) What is possessed? [search for an object later on] 

Th2 next three words, "3 of them" form a noun group with an explicit 
reference to a previous set. In short, the "3" causes EDUCE to (i) enter noun 
group mode; (ii) instantiate a conceptual quantity; and (iii) generate a request 
to find an object-concept in this noun group. In the context of a quantified 
noun group, the word "of is interpreted to mean that the conceptual referent 
to follow (currently unknown) is the whole which possesses as a part the 
conceptual object in this noun group. In terms of requests, the word "of" 
expects a definite reference of a conceptual object to follow and if that object is 
found, the quantified object in the current noun group is "part of" a 
previously known set. The word "them" leads to a set reference of the 
previous set of Jacob and Kathy's 8 soda cans. The noun group "3 of them" is 
eventually merged into an "object-concept 77 with a quantity of three which is 
PART-OF a previously known set. The outstanding request (i) from the word 
"has" which is expecting an object is now satisfied. The can-object is merged 
into the OBJECT slot of the (2) STATE-concept. 
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A final conceptualization for the entire sentence resides in STM: 

CONCEPTS: (2) STATE: possession 

POSSESSOR: ((+human) (DEFINITEref: Jacob)) 
OBJECT: ((physical (-animate) 

(function: contain(object: soda)) 

(quantity: 3) 

(PART-OF: "previous set containing 
this object"))) 

Of specific importance in this sentence is how EDUCE represents the explicit 
"part of" reference generated by the "of them" wording. This sensitivity to 
slight changes in problem wording highlights the importance of "starting 
from the beginning." As discussed in this body of the paper, such explicit 
wordings and their associated representations facilitate the processes of text 
integration which lead to a coherent situational representation of the 
relationship between sets. As a counter-example, a more traditional wording 
of the second sentence in this type of problem is: 

'Jacob has 3 soda cans." 
In this case, EDUCE would encounter no explicit set reference and thus would 
not be able to represent any connection between the two sentences. 

As shown in the detailed parse of sentence two, EDUCE generates a PART-OF 
slot from the "of them" phrase and thereby indicates an explicit connection 
between the objects in the first sentence and the objects in the current 
conceptualization (second sentence). SELAH performs the following steps in 
order to integrate this new conceptualization from EDUCE with the previous 
set in LTM-(l). First, SELAH notes that the source of the objects in the second 
sentence is a previous concept, that is, the objects from the first sentence. 
SELAH searches LTM for the set which includes the "soda-can" type object. 5 
Finding the previous set of eight specific "soda can" objects, SELAH knows 
that the source of Jacob's three cans is a unique quantified set (as opposed to 
the universal set of all soda cans). Given that the source of the three cans is 
the set of eight cans, SELAH associates this qualitative relationship with the 
mathematical relationship that the set of three is a "member of" or "part or" 
the set of eight. SELAH makes a new set of three in LTM-(2) and links the 
two sets together, i.e., the set in LTM-(2) is PART-OF the set in LTM-(l). 



5 In this example, the reference from the word "them" insures that the two sets being 
integrated involve the same type of object. In general, when SELAH attempts to determine if 
two sets are related, the objects in those sets need not exactly match. SELAH performs the 
following constraint ( hecks: do the objects match exactly (e.g., cans and cans) or are they "like- 
types" (e.g., infer that cats are pets). If the objects are an exact match (e.g., cans and cans), is 
one object more qualified than the other (e.g., pepsi cans and cans) or are they equally qualified 
(e.g., big black cats and big black cats). o ^ 




26 



AERA-94 



LTM-(2) 

STATE: possession 

POSSESSOR: (same Jacob as above) 
SET: 

OBJECT: (physical (-animate) (function: contain(object': soda))) 
QUANTITY: 3 

PART-OF: the SET in LTM (1) 

Having successfully integrated the two conceptualizations, SELAH infers that 
this static PART-OF connection can be associated with a procedural situation 
where one set is SEPARATED-FROivi another set. More specifically, because 
LTM-(l) is the source of objects in LTM-(2), the set in LTM-(2) can be 
"constructed" in a manipulative sense by separating three cans from the 
previous set of eight in LTM-(l). The transition from statically integrated text 
to mental actions such as separating-from is a critical feature of SELAH's 
novice problem solving process. The hypothesis is that novice problem 
solvers who are working without the aid of manipulatives continue to think 
of relationships between sets in terms of the actions upon those sets. 
Theoretically, the model proposes that young children must first construct 
procedural or action-oriented representations in order to arrive at an 
arithmetic operator (For similar arguments, see Fuson (1994b); Reusser (1990); 
Stern & Lehrndorfer (1992) and Weber-Russell (1994)). While some problems 
include explicit actions (e.g., giving) which facilitate children's success (c/. 
Briars & Larkin, 1984), many word problems do not contain action language 
and require additional processing to infer a procedural interpretation of 
statically expressed relationships. In the current example, SELAH infers a 
separating-from interpretation of a static part-of relationship between two 
sets. 

The third sentence involves a similar link between the third sentence and the 
first, i.e., Kathy's are PART-OF the set of eight. In addition, the word "rest" 
implies that a separating-from action has already occurred, that is, the "rest" 
are left behind. Thus, the SEPARATED-FROM action that was inferred in 
the previous sentence is reinforced in this sentence and the objects remaining 
from that action are explicitly marked as belonging to Kathy. In short, 
SELAH's current representation is: 

LTM-2 SEPARATED-FROM LTM-1 resulting in LTM-3. 

In addition to making sets and performing text integration, SELAH monitors 
some of its cognitive processing such as the number of sets to remember and 
the number of inferences that must be made. Table 1 summarizes some of 
SELAH's processing which led to this current representation. 
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The question in the last sentence focuses SELAH's attention on the unknown 
amount of Kathy's set. At this point, SELAH's representation has reached a 
level of abstraction where only the quantities and arithmetic action are of 
interest For example, SELAH is no longer concerned with Kathy and Jacob as 
participants or soda cans as the type of objects. Focusing on the quantities and 
arithmetic action, SELAH selects an appropriate counting strategy, in this case, 
Counting-Doion-From (Carpenter, 1985) to arrive at an answer of 5, e.g., 

"Ok, start at 8: 7 that's 1, 6 that's 2, 5 that's 3; the answer is 5." 



Table 1 

Summary of SELAH's processing for 
first three sentences of Combine 5 



Jacob and Kathy have 8 soda cans. 
Jacob has 3 of them. The rest are Kathy's. 




Sentence 




1 


2 


3 


Average 


Processing 
Summary 


make set 


explicit part-of; 
infer SEP-FR 


explicit part-of; 
reinforce SEP-FR; 
explicit result of 
of SEP-FR 




Concepts in 
Memory 


Q & K's 8] 


[SEPJ'sS] 
[FRJ&K's 8] 


[SEPJ's 3] 
[FRJ&K's 8] 
[REsult K's ?] 




# of concepts 


1 


2 


3 


2.00 


# of inferences 


0 


1 


0 


0.33 



::2 



9 

ERJC 
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